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MECHANICAL ERRORS ON TEACHERS' 
EVALUATION OF THE NONMECHANICAL 
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Black, Janet Knecht 
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D'Angelo, John Louis 


PREDICTING READING ACHIEVEMENT 
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UNIVERSITY: 1944-1976 


Infantino, Robert Louis 


TESTING AND ACCOUNTABILITY: A 
SURVEY OF THE KNOWLEDGE AND 
ATTITUDES OF NEW YORK STATE 
SECONDARY SCHOOL ENGLISH TEACHERS 


Johnson, Nancy Wehrle ' 


THE USES OF GRAMMATICAL AND 
RHETORICAL NORMS, PEDAGOGICAL 


_ STRATEGIES, AND STATISTICAL 


METHODS IN DESIGNING AND 
VALIDATING A COMPOSITION 
PLACEMENT INSTRUMENT 


Mead, Nancy Alexander 


THE DEVELOPMENT OF AN INSTRUMENT FOR 


ASSESSING FUNCTIONAL COMMUNICATION 
COMPETENCE OF SEVENTEEN YEAR OLDS 
Newcomb, Joseph Scott 


THE INFLUENCE OF READERS ON THE 
HOLISTIC GRADING OF ESSAYS 


- 


Reese, Linda Lee . 
; STUDY TO DETERMINE THE CONSTRUCT 


' VALIDITY OF THE GRAMMATIC CLOSURE 


SUBTEST AS A MEASURE OF ‘LANGUAGE 
ABILITY USING SIGNAL DETECTION THEORY 


" Richards, Roger Allan 


THE DEVELOPMENT AND EVALUATION OF A TEST 


_ OF CRITICAL LISTENING FOR.USE WITH ~~ 
COLLEGE FRESHMEN AND SOPHMORES 


Shank, Kathlene Stapfer 


‘TRAINING PRE-SERVICE TEACHERS IN USE OF 
STANDARDIZED LANGUAGE ARTS ACHIEVEMENT © 


“ 


. TEST: AN EXPERIMENTAL PROJECT 
' Simpkins, Charlesetta 


EFFECTS OF THE CULTURAL CONTEXT OF 
LANGUAGE ON THE COGNITIVE PERFORMANCE 

OF BLACK STUDENTS 

Singleton, Doris Jacobs % 


THE RELIABILITY OF RATINGS ON THE 

ESSAY PORTION OF THE LANGUAGE SKILLS 
TNATION 

Sutton, Gary Alan 

EVALUATION OF FLORIDA SECONDARY ENGLISH 

TEA S 


Wynn, James Howard 


. DETERMINING THE INTERNAL “CONSISTENCY. OF 


‘ENGLISH COMPOSITIONS USING SELECTED 


* CRITERIA 


A STUDY OF THE EFFECT OF SELECTED MECHANICAL 
ERRORS ON TEACHERS’ EVALUATION OF THE NON- 
MECHANICAL ASPECTS OF STUDENTS’ WRITING 


ALLEN, Charles Lee, Ed.D. 
Temple University, 1976 
“— 
The purpose of this study was to investigate the extent to . 
which the presence of mechanical errors in students’ writing 
affects tedchers’ evaluations of the non-mechanical aspects of 
writing. The specific questions investigated in the study 
erned: (1) the effect of errors in spelling, punctuation, 
capitalization, and sentence structure on teachers’ evaluation 
of the quality, organization, and development of an idea and 
the individual style of the content of students’ writing; (2) the 
relative effect of such errors on junior high school teachers 
as opposed to senior high school teachers; and (3) the rela- 
tionship of the varied aspects of students’ writing to each other. 


s 


' 


- Procedures 


The subjects of this study were 412 teachers of secondary 
school English in a large urban school district. The 42 schools 
involved in the study were randomly ‘assigned to one of four 
groups so that each group was composed of a cross-section of 
the district’s schools. 

Following the identification of a typical sample of uncor- 
rected student writing, the investigator reworte the sample in 
three different versions. In the first revised version, all me- 
chanical errors were removed. For the second version, the 
etudent’s original mechanical errors were retained, and sev-- 
eral additional ones were supplied by the investigator. ‘For 
the third version, all mechanical errors were retained, .and 
syntactical patterns of a variety of non-standard English were 
added by the investigator, In all four versions of the writing 
sample, the word choice of the student was the same. All four 
versions of the writing sample were assigned to one of the four 
groups of teachers. 

Each group of teachers was requested to evaluate the para- 
graph assigned to it using a 1-7 rating scale for each of eight 
categories. Although four of the categories concerned the me- 
chanics of writing and four corcerned the content of writing, 
this division was not brought to thé teachers’ attention. 


Findings 
‘The results of thig study have been summarized in terms 
of the three hypotheses of the investigation. First, there were 
no significant differences among the mean content ratings for 
the four versions of the writing sample, between the ratings of 
jynior and senior high school teachers, or among the interac- 
tion of the main effects. There were many significant and 
highly significant correlations, as determined by the Spearman 
rank correlation coefficient (rho). Among the most consis- 
tently significant correlations were those: (1) between the total 
subscore for mechanics and the total score assigned to the - 
writing sample, and (2) between the total subscore for content 
and the total score assigned to the writing sample. On all four 
versions of the writing sample these correlations were all sig- 
nificant at the .001 level. 
Conclusions aS 


Subject to the mitations of the study, the following conclu- 


_ Sions were drawn: 1. Many teachers are already able to ignore 


r 4 . 


{ % 

the presence of mechanical errors. when they are evaluat- 
ing only the non-mechanical aspects of students’ writing. 

2. Teacher-training institutions and school districts need to. 
identify such teachers, determine the criteria théy use in eval- 
uation, and design programs | to assist other teachers to eval- 
uate writing more effectively. 3. Students who write in a dia- 
lect significantly different from Standard English do not suffer 
unjustly because their writing’is judged against the conventions 
of edited written Standard English. 4. Junior high school teach- 
ers are less likely than senior high school teachers to: (1) per- 
ceive mechanical errors in writing, or (2) be negatively af- 
fected by the presence of mechanical errors, 5. The length of 
time teachers have been teaching, the number of graduate hours 
of study in English, and the number of graduate hours of study * 
in education have no relationship to their ability to evaluate  / 
eitherthe $ pechanics or the content of students’ writing. 
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FORMAL AND INFORMAL-MEANS OF ASSESSING THE 
COMMUNICATIVE COMPETENCE OF KINDERGARTEN 
CHILDREN 


BLACK, Janet Knecht, Ph{D. 
The Ohio State University, 1977 


Adviser: Professor Martha b. King 


The assessment of kindergarten children’s communicative . 
competence was examined with regard’to two formal instru- 
ments (the Illinois Test of Psycholinguistic Abilities and Circus | 
10) and the informal énvironment of the sociodramatic area. 

Communicative competence, as defined in this investigation, 
has two aspects (1) grammatic competence, and (2) interac- : 
tional competence. Grammatic competence was measured by 
the kindergarten child's syntactic maturity, vocabulary diver-- 
‘sity, ang use of irregular inflections in the three oral language 
samples: 

The second phase of the investigation concerned the assess- 
ment of intefactional competence. . Pinnell’s (1975) categoriza- 
tion of the functions of language was applied to the three eval- 
.uative approaches to determine the number of functions elicited 
by each assessment techniqué. The seven functions include In- 
strumental, Regulatory, Interactional, Personal, Imaginative, 
Informative, and Heuristic. In addition, oral language samples 
collected from the sociodramatic area via two tape recorders, 
a video tape, and the researcher’s notes were analyzed accord- 
ing to the interactignal competency checklist. The interactional 


%. gompetency checkif8t was designed by the researcher and was’ 


baséd upon the thought and research of Cicourel, (1972); Hymes, 


. (1971); Lewis and Cherry, (1975); Mishler, (1976); Ritchie- 


Key, (1976); and Schegloff and Sacks, (1973). The interactional 


* competency checktist consists of four categories: (1) the ability 


to adapt to changes in the setting, (2) nonverbal appropriateness, 
(3) familiarity with normal constraints and conditions of con- 
versation, and (4) sequencing. 

Analysis of the data suggests that the formal means of as- 
sessment used in this-investigation provided limited and inac- 
curate mformation about a child’s grammatic competence, and 
provided no information regarding the child’s interactional 
competence. However, the sociodramatic area generally pro- 
vided more Compre*ensive information about the child’s total 
communicative competence. . 

In terms of vocabulary diversity, the sociodramatic area 
provided kindergarten children with the opportunity to use a 
significantly greater (.001 level) variety of words than either 
of the two standardized test situations. The natural environ- 
ment interview indicated that children could use irregular noun, 
verb, and pronoun inflections to a significantly greater degree 
of accuracy (.001 level) than the two standardized tests sug- 
gested. There was no significant difference between the three 
oral language ‘samples in terms of the analysis of syntactic 


5 complexity. 


The second phase of the investigation concerned the inter - 
actional aspect of communicative competence. The nature of 
the gociodramatic area provided the opportunity for the subjects 
to use six of the seven functions of language while Circus 10 
and ITPA each elicited only one function or use of language. - 
This.study also determined that the sociodramatic area:of the 
kindergarten classroom provided an environment in which the 
researcher was able to document numerous examples of the 
subjects’ interactional competency as categorized by the inter- 
actional competency checklist. 

This study provides evidence that children have virtually 
obtained grammatic and interactional competence by the time 
they enter school. In short, if children are given the dpportun- 


‘\ ity to interact in envirbnments over which they exercise some 


1: ¢ontrol, their response tends to be more natural and reflective 
ee 


of theic true communicative competence. These findings raise 
some questions for educational practice concerning the (1) com- 
plete dependency upon standardized tests in assessing language - 
performance, (2) the “deficit” approach used in evaluating chil - 
dren's oral language, and (3) the narrow approach of language 
development programs. ‘It would appear that the opportunity 
to practice and use language with peers in the natural envtron- 
ment could be the most effective language program, 
In conclusion, this study suggests that the assessment of 
‘communicative competence, if it is to be reliable, valid and 
comprehensive, must include interactional as well as gram ¥ 
* matic competence. This recommendation is indicative of the SY 
trend in language*research which takes into consideration a 
more global kind of competence necessitating semantic anal- 
ysis in the social setting. Order No. 711-24,598, 221 pages. 
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~ PREDICTING READING ACHIEVEMENT IN A SENIOR 
HIGH SCHOOL FROM INTELLIGENCE, LISTENING, 
AND INFORMATIVE WRITING 


D’ANGELO, John Louis, Ed.D. 
University of Pittsburgh, 1977 


* Adviser: Dr. Harry W. Sartain 


The purpose of this study was to develop a regression for- 
mula which could he used to predict the reading achievement 
of high, average, and low ability ninth grade students from 
knowledge of mental age and selected skills in informational 
writing and listening. A secondary purpose was to determine 
the inter-rater reliability of an assessment instrument for 
informational writing. 

The data-collection instruments employed in this study 
were the Otis-Lennon Mental Ability Test, J and K; the Brown- 


Carlsen Cistening Comprehension Test; the Listening to Re- 
member Specification Test, LeVel E; and the Informative Writ- 
ing Eeale, vanced Middie School Level. 

sample total of 245 students was randomly, selected from 


the total ninth grade population of a suburban Pittsburgh school 
district. The 245 students were categorized as possessing 
high, average, or low ability on the basis of the mental ability 
test. Thus, 102 students were selected for the high ability cate- 
gory, 120 were selected for the average ability category, and 
23 were selected for the low ability category. The various mea- 
sucing instrument scores o/ 77 students in the high category, 
90 students in the average category, and 18 students in the low 
category were utilized in the development of a regression for- - 
mula fer that category. The remaining students in each cate- 
gory served as a cross-validation groyp to determine the ef- 
fectiveness of each equation. i j 

After the administration and scoring’of the various instru- 
ments, correlation coefficients were obtained to dgtermine the 
relationships between reading achievement and the independent 
variables, Multiple regression techniques then were employed 
to obtain predictive formulae and a stepwise technique was ~ 
used to determine the best predictors. The results of the form- 
ulae then were compared with actual reading scores to ascer- 
tain the effectiveness of each formula. . 
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7 Lennon Mental Ability 
re 


. 
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“all correlation coefficients bed the ie aie Neeonemg 
.O1 level of con’ ce - 

Locale nant cant having the largest correlation with 

achievement, . 1, However, none of the multiple 

“regression formulae which were {ound to be most effective 

used the mental ability variable. All of the formulae which 

were found to be most:effective used a measure of listening 
ehension and/or listening memory, * — 

<onThe regression formula developed for the‘total main sam- 

ple was found to be morg convenient and less time-consuming 

‘to use than’the other formulae which were developed because ; 

students did not have to be grouped according to ability. Soler 
formula, which employed student scores on tie eee 
comprehension, listening memory, and mechanics —_ J 
the writing scale, was found to have a 79.5 percent 0 pe 
ment and a 20.5 percent of error when compared with ac 
cores. . 
a raavensisu formyla developed for the low pre trad . 
employed only the listening memory score and was aah 2 
have an 89 percent of agreement and only an Bt Lag lant = 
when compared with actual reading scores. This nae Dee 
recommended for populations which have been grouped ace 
bility. ; 

. eee of Variance technique employed to etecerol if 
a Significant difference existed among three ole caries 
Informative Writing Assessment Instruner eg ed no 

ifi T of confid ; 
nificant difference at the .01 riers er clk, 1k pigel: 
1) 
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THE, STUDENT USE OF ENGLISH EXAMINATION AT 
NOKTH TEXAS STATE UNIVERSITY: 1944-1976 


DUNCAN, William Neil, Ed.D. ‘ 

North Texas State University, 1976 ‘ 
‘ This study is concerned with the history of the Student Use 
of English examination gt North Texas State. University in 
Denton, Texas. Conceived in 1944 and implemented in 1946, 
the examination serves as a measure to insure that marginal 
English students demonstrate a minimal proficiency in com- 
position evidenced by a three and one-half page essay written 
on topics related to their major fields. The topics, changed 


. 


annually, are not disclosed until the time of the supervised Pe 


emamination. A marginal student, one who has made any grade 
of D in the required four courses in English, must be judged 
“salisfactory” to graduate from the university. 

The examination has undergone four phases, from one in 
which it was required of all students to the present one in which 
it is taken by marginal students only. Records show thet no 
less than 22,403 students have written for the examina since 
its inception. | 

Analysis of the marked errors on 971 examinations dis-, 
approved since 1962 reveals a mean of twenty errors on each 
composition, with spe}ling and commas constituting 9.4 errors, 
No failing paper contained less than eleven cited errors. 

An analysis of five hundred questionnaires returned from 
former students and one. hundred questionnaires from current 
students indicates the following findings: (1) the largest per- 
centage «f students taking the examination is from the College 
of Education, but a larger percentage of failures comes from 
the College of Business; (2) 48.4 per cent of the formef stu- 
dents and thirty-three per cent of the current students spent no 
time in preparing for the examination; (3) some sdeint 
that others passed\the examination dishonestly; (4) fiff}seight 
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nt of the current students and 54.4 per cent of the former 
ceohesita feel that the requirement does nothing to enhance the 
reputation of the university; (5) 45.8 per cent of the former 
- students and fifty-five per cent of the current students favor 
abandoning the requirement as a prerequisite to graduation; 
(6) although each of the respondents made at least one D in 


English, twenty-five per cent state that they do a greht deal of — 


in their current employment, the largest number of 
pariah now epgaged in teaching; and (7) only twenty per 
cent of the respondents currently have secretaries. 

The major conclusions of the dissertation are that no im- 
provement in writing skills is revealed on the failing examina- 
tions for a fourteen-year period, and that 49.7 per cent of the 
surveyed students feel that the examination is not a learning 
experience. Though no specific criterion exists for evajuation 

* of examinations, the graders have been consistent tn their 
ng of errors. |. 
pai i mecca or to include the implementation of a new 


phase in which only students with a cumulative English average 
of less than C be required to take the examination and a recom- 


mendation that the freshman English program place more em- 
phasis upon writing. In addition, ‘a recommendation is made to 


encourage professors in other disciplines to stress more writ- 


r subjects. 

et eae significance of this dissertation is that it 
assembles all extant historical documents concerning the ex- 
amination in one place, it determines a criterion. for failing 
examinations through an analysis of 971 disapproved essays, 

it records evaluative responses from six hundred former 
and current writers of the Student Use of English examination 

niversity. ‘ 

nove a aia No. 77-11,102, 307 pages. 


G AND ACCOUNTABILITY: A SURVEY OF 
EDGE AND ATTITUDES OF NEW YORK ATATE 


” SECONDARY SCHOOL ENGLISH TEACHERS 


. 


INFANTINO, Robert Lovis, Ed.D. 2 
" State University of New York at Buffalo, 1976 


\ ; Problem 


e . / b ’ 
This study was conducted to describé the knowledge and 
attitudes of secondary school English teachers in New York 
State toward the uses of externally prepared examinations, 
including standardized tests and statewide Regents examina- 
tions, and toward the effect that accountability programs are 
having on teachers, school systems, and pupils, especially a¢- 
.countability programs which use standardized testing of pupils 
as a measure of teacher performance. Twelve research ques- 
tions were posed for this study. - 


, 


Procedure 


A random sample of 985 public school secondary English 
teachers was drawn from all school systems in New York 
State, using the computer resources of the New York State 
Education Department. These teachers were mailed a survey 
questionnaire containing thirty-seven items concerned with the 
teacher’s knowledge of current testing practices. An addi- 
tional twenty-seven questionnaire items wert attitudinal, using 
“a five-point Lickert-type scale. On these items teachers were 
asked to indicate their attitudes toward testing and accotnt- 
ability. ears 

A total of 497 useable questionnaires were returned, a 
60.5% rate, Follow-up interviews were conducted with ten 
randomly selected respondents from the Western New York 
area, - 


, : Findings 
Descriptive data were compiled using the program package 
SPSS. Some of the major findings were: 1. There is a wide 
variety of all types of standardized tests being administered 
; yee Z 


sary) 
‘ 


\ 
« 
5 
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in New York State. 2. Tests results are accessible|to teachers 
‘and are used appropriately for guidance and placenjent pur- 
poses; teachers are seldom forced to help student prepare 

for standardized tests. 3. Younger, less experienced teachers 
feel restricted in both content and method by externally pre- 
pared examinations such as the Regents Comprehensive English 
Examination, Teachers showed a lack of familiarity with 
testing terminoldgy and test construction. They dill seem to 
recognize problems inherent in standardized testing such as 
cultural bias and unreliability of a single test sco 
performance on standardized or externally prepar¢d examina- 
tions does not affect teacher evaluation, promotion; or tenure. 
6. Accountability programs are not widespread in New York 
State and do not Sppear to be having much effect 
or pupils. 7. Teachers expressed a willingness to 
countable for those factors over which they have s 
such as knowledge of subject matter. They aren willing to 
be held accountable for bringing about growth in pypil learning 
based on standardized test scores. ; 


Recommendations 


* Selected recommendations based on the sur 
view data are these: 1, School districts in New 
should review testing practices to be sure that the standardized 
tests being administered are adequate in regard fo the present 
curriculum, the population of the district, and the usefulness 
of the test results to the district. Tests which are not adequate 
should be discounted, 2. Teachers should not unnecessarily 
restricted in content or methodology in English plasses be- 
cause of externally prepared examinations. District-wide and 
State-wide final English examinations should be eliminated, 

3. Teachers should not now be evaluated on the basis of how 
well or poorly their students perform on standardized tests or. 
externally prepared tests because of inadequate knowledge 
about'the relationship between teacher performance and pupil 
learning. 4. If accountability programs are to be introduced 
into school districts, the programs should be ‘designed for the 
improvement.of instruction and not primarily to save money 
or dismiss teachers. 5, School districts must provide better 
information regarding the purposes and practices of the dis- 
tricts’ testing programs, especially to younger, less experi- 
enced teachers. 6. Professional organizations like the Na- 
tional Council Teachers of English and the/New York State - 
English Council need to continue to keep teachers informed 
about the use and misuse of standardized Aesting and about the 
accountability movement in education, i 


Order No. 77-6143, 224 pages, 


THE USES OF GRAMMATICAL AND RHETORICAL NORMS, ® 
PEDAGOGICAL STRATEGIES, AND STATISTICAL METHODS 


IN DESIGNING AND VALIDATING A COMPOSITION PLACE = 
MENT INSTRUMENT 


. JOHNSON, Nancy Wehrle, D.A. 


The Catholic University of America, 1977 


The use of standardized tests is now being challenged, The 
most serious question raised is whether such tests actually 
measure students’ knowledge relative to a particular subject 
matter or reflect nothing more than an ability to do well on a 
particular kind of test. Grantihg that these formulations are 
extxemes, the issue they focus is real, Are standardized tests - 
valid? The dissertation takes up the problem from the point 
of view of English composition. Specifically, it,is concerned 
with the design of a placement test for a two-year college, but, 


" if the design itself.is sound, it may, with appropriate modifi- 


7 


, 


cation, serve as a model for other institutions, 


( 


‘ 


Chapter I examines the history of placement testing in En- 
glish composition from 1900 to the present. Two conclusiois 
emerge from the survey, First, where rigorous controls are 

. established in their administration and grading, essay tests = 

that is, actual student writing samples — may provide mate- 

+ rials for a standardized test. Second, the use of these mate- 

rials in the construction of a standardized test needs to be 

guided by norms which are public and verifiable, 

Chapters II, III, and IV discuss pedagogical and subject 
matter considerations, The primary pedagogical considera- 
tion is whether the test can provide an accurate index of stu- 
dents’ ability to compose grammatically correct and rhetori- 
cally effective, sentences. Chapter III outlines procedures 
for determining what constitutes grammatical correctness, 
addressing itself specifically to the relation between the 
grader’s or test designer’s conception of grammar and his 
writing of questions. Chapter IV treats the relation between 
the conception of rhetoric one holds and the writing of ques- 
tions. Both Chapters II] and IV give examples of questions 
based upon students’ writing samples, The contention of these 

, two chapters, and ‘gt the chapter on pedagogy, is that clArifi- 
cation of ideas leads to greater precision in practice, preci- 
— sion that is reflected in the identification of strengths and 
weaknesses in actual student compositions and in the use of 
those compositions as materials for the construction of a 
placement instrument, 
Chapter V is a handbook composed primarily of statistical 

. analyses. Among the topics discussed are field testing, va- 
lidity, and reliability. The handbook is also accompanied by 
directions for administration and scoring, Chapter VI pre- 
sents follow-up studies which indicate that the test has proved 

a valid and valuable instrument, Chapter VII offers conclu- 

’ gions/which relate the present test to a booklet recently pub- 
lished by the National Council of Teachers of English. The 
bpabias, Carin Sunes. el Testing da eink, Ors Weide 
lines for designing standardized tests; the test described in 

s ‘the dissertation anticipates almost completely the recommen- 

. dations of the committee representing NCTE, j 
4 Order No, 77-11,039, 172 pages. 
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, THE DEVELOPMENT OF AN INSTRUMENT FOR ASSESS-= 
ING FUNCTIONAL COMMUNICATION COMPETENCE OF 
SEVENTEEN YEAR OLDS 
MEAD, Nancy Alexander, Ph.D. 

‘University of Deaver, 1977 


y The purpose of the present study was to develop an jnstru- 
ment for assessing the functional communication competence 
of seventeen year olds. The instrument was designed to be 
used in the National Assessment of Educational Progress, a 
national survey which assesses the performance of elementary 
and secondary students with respect to specific educational 
objectives, 

The characteristics of the National Assessment influenced 
the conceptual and methodological approaches used in the 
present research. The study adopted a functional approach 
to the concept of communication ability. This conceptual 


framework focused on the accomplishment ‘of communication 
purposes in social situations, The study used an objective- 
referenced approach to measurement. This method was imple- 
mented in six steps: objectives development, item development, 
field testing, item analysis, item selection, dnd tests of reli- 
ability and validity. . 


x 


‘ 


The study generated three products: (1) selected items, 
(2) tests of reliability, and (3) tests of validity. The first 
product was a group of items which assessed four functional. 
categories (informing, controlling, sharing feelings and ritual- 
izing) and related communication attitudes. The four functional , 
categories were further divided into speaking (and nonverbal = « 
exp?essing) and listening (and nonverbal recognizing) perspec- 
tives. Three packages of items (each including approximately 
35 minutes of testing) were selected. The selected items var- 
fed from category to category in terms of numbers, dffficulty 
level, discrimination power, and racial/ethnic bias. The se- 
lected items are described but not included in the dissertation. 
The National Assessment requires that their materials be kept 
secure. ‘ ¢ 
The second product of the study was tests of reliability of 
the selected items. Each category of selected items was tested: 
for internal consistency. All of the categories either met or 
approached an acceptable level of reliatjlity (criterion = ,70) ] 


_ except the two sharing feelings categories. 


a 


The third product of the study was tests of the content valid- 
ity of the selected items and construct validity of the categor- 
ies. The content validity of the selected items varied from 
category to category. All of the selected items appeared to be © 
an accurate measure of the objectives (face validity)., How-' 
ever, some groups of selected items appeared to sample the 
domain of their respective categories better than others. 

The construct validity of the category system was explored 
by a number of statistical comparisons. Correlations were 
computed between subtests scores which réflected the follow- 
ing relationships; (1) between speaking and listening, (2) be- 
tween informing and controlling, and (3) between knowledge 
and related attitudes. No strong relationships were identified, 


* This indicated that the assessment categories were distinct. 


However, the variety of measurement strategies used in the 
(direct, indirect and affective) may have contributed to 
ure to find significant correlations, 

The results were discussed in relationship to the nature == * 
of the content area, functional communication competence, and 
the nature of the survey effort, “a lar¢1-scale cssessment of 
specific objectives. It was difficult to assess the functional \. j 
areas of sharing feelings and ritualizing because they are “s 
closely tied to caltural and socio-economic norms. It was - 
difficult togstablish reliability and validity because the objec- 
tive-referenced approach favors the development of a limited 
namber of items which accurately reflect specific objectives 
rather than a large group of items which differentiate levels 
of performance within a general,content domain. 

: Order No, 77-27,426, 183 pages. 


THE INFLUENCE OF READERS ON THE HOLISTIC GRADING 
OF ESSAYS » 


‘ 


NEWCOMB, Joseph Scott, Ph.D. 
The University of Michigan, 1977 


Chairman: LeVerne S. Collet 


This study examined the effects of readers and their geo- , / 
graphic region, sex, and race on the holistic grading of essays.. 
Four hypotheses were stated for investigation. Hypothesis One 
sought to determine whether readers would account for differ- 
ences in mean essay grade assignment, and variance of essay | 
grades and whether or not a satisfactory level of inter-reader . 
reliability would be achieved. Hypothesis Two concerned the ‘ 
effects of the reader region, sex, and race dimensions. Hy- 
potheses Three and Four considered the dimensions shafted 
' between reader and writer and the affect on mean grade as- 
signment. 2 8 s 


; 
! 


a ¥ 

Readers and essay writers were identified by region, sex, THE DEVELOPMENT AND EVALUATION OF A TEST OF 
and race, Readers were trained in the holistic grading method CRITICAL LISTENING FOR USE WITH COLLEGE FRESH- 
and graded the entire sample of 192 original papers. MER AND SOPHOMORES 

Analysis of variance was used to test the main effects of 

* peaders, their region, sex, and race and the interactions of RICHARDS, Roger Allan, Ph.D. 
these reader dimensions with the same essay dimensions and New York University, 1976 
the relationship between holistic essay grades and the number sie v*, 
. and type of dimensions on which readers and essay writers Chairman: Professor Neil M. Postman 
were the same. The Hartley F,,a, test was used to evaluate 
the effect of readers on the variance of"essay grades. The The‘purpose of this investigation was to develop an objec- 
average inter-reader correlation coefficient and 95 percent ‘ tive test of critical listening ability for use with college freah- 
confidence limits were used to judge the adequacy of inter- men and sophomores and to evaluate its impact on the teaching 
reader reliability. of critical listening at the college level, 

Readers (considered as a random variable) differed signifi- The' technique of content analysis of professional literature 
cantly in the severity with which they assigned holistic grades was used to develop an sis of critical listening to serve 
and the extent to which they made use of the entire range of as the basis for the spedifications of the test. This approach 
the grading scale. The average inter-reader correlation co- - was based upon the premise, supported by research, that criti- - 
efficient (.72) was significantly less than the criterion (.80) cal listening is not a single, holistic skill but rather a com- . 
adopted for this study. No significant relationships were found posite of subskills. Tabulation of frequency of use of key terms 
between holistic essay grades and reader region, sex, and race in the literature on listening, critical listening, and critical 
or between holistic essay grades and the number or type of thinking, coupled with the researcher’s synthesis, led to the 
dimensions shared between readers and writers. Considerable ~ following statement of test objectives: 
mean essay grade difference was observed between male and , The Critical Listening Test will measure the abilities to 

. female and black and white seventeen-year-old writing skill (1) Identify assumptiond and inferences, (2) Recognize main 
* - during the course of data analysis. Female writers received ideas and arguments, (3) Identify fallacies in the use of data, 
' higher grades than male writers and white performance was (4) Judge relevance of data, (5) Recognize bias in persuasive — 
rated higher than that for black writers. . discourse, (6) Distinguish between statements of fact and state,’ 

‘Analysis of the data assuming readers as a fixed effectre- ° ments of opinion, and (7) Hecngnies errors in deductive rea- 

vealed significant differances among indiyidual reader grading soning. 
-. practices leading to an ability to predictAfheir tendency tobe - A technique was developed to measure listening without the 
strict or lenient graders on the basis of their geographic re- risk of contamination from requiring students to read. The 
. Bion, sex, and race and most combinations of these three ~ , researcher adopted a modified multiple-choice item in which 
classification.variables. . a stem was followed by two options, either, both, or neither of. 

. Regardless of the assumption made concerning reader ef- which. might be correct. It was found that students were able 
fect, it appears from the results of this study that the holistic to follow a single reading of such items and to respond without 
essay grade which a writer receives is attributable in a large confusion when neither the questions nor the responses were 
measure to the reader who assigns it rather than to the writer’s supplied in printed form. 
ability. Order No. 77-18,082, 168 pages. A preliminary form of the test was administered for the 
j F purposes of time analysis and try-out of adniinistration - 
. e procedures. A split-half reliability of .76 was achieved, Items 

’ found to be nondiscriminating were revised for the final ex- 
perimental form, which was administered by tape. The tape 
consists of eight passages, each read by a different person, 
with the investigator reading directions and test questions, 

, The final experimental form of the Critical Listening Test 
- ‘ was administered to 16 classes at an urban New England com- 
A STUDY TO DETERMINE THE CONSTRUCT VA LIDITY munity college. It was possible to include 186 students in the 
@ OF THE GRAMMATIC CLOSURE SUTEST AS fa MEASURE evaluation of the test. Claims of validity were based on a cor- 
OF LANGUAGE ABILITY USING SIGNAL DETECTION _ relation of .47 with the Watson-Glaser Cyitical Thinking Ap- 
THEORY : raisal (n=40), .64 with Verbal component of the Scho! 
Aptitude Test (n-57), .47 and .41 with the McGraw-Hill Read- 
 RERSE, binds bee, Ee 0, , ie Test (n=90, n=57), and .49 with Brown-Carlsen Test 
D&lahoms State University, 1976 ae omprehensiod (n=17) 
\ Ee or 38 .63 between subskill score t 
Scope of Study: The purpose of this a(ucy ste $9 determine ; ene er wees oie se Halt of .02 to .23 ae 
the construct validity of the Grammatic Closure subtest ofthe. of pubskilia were interpreted as supporting the hypothesis 
ITPA as a measure of language ability. Signal Detection Theory that eritieal listening consists of at least the seven subskills 
sensor and as a decision maker. One hundred and thirty-two The drop in reliability from .76 (o.8 between the pretimi- 
Ls Eastern Oklahoma Head Start children were administered the 
. nary and final experimental forms raised the possibility that 
Standard version of the subtest, a Signal Detectionversion, and use of different voices may have been distracting to students 
~ , a Restatemeht version of the same subtest. ; and thus interfered with the measurement of critical listening. 
. Findings, and Conclusions: Results were analyzdd using Low internal-¢onsistency reliability was not regarded as a 
Pea *s Product-Moment Correlation Coefficients with .05, source of weakness of the'@fitical Listening Test, since it is 
as the acceptable level of coniidence, The-conclusions a heterogeneous = me eee a homogeneous, fatcorenk 
re that the study was heavily confounded by the subjects’ Since the results obtained agreed with other findings that 
naeney to comply on the Signal Detection seraton.. In epite college students are not highly skilled in listening, the investi- 
’ of this, there was modest support for the;yalidity of the Gram- gation supported the claim that critical listening should be / 
of matic Closure subtest as a measure of 4anguage performance. taught. It was: suggested that the availability of a means of Pa 
';" Further, there was support for the fact that measuring lan- measuring the stills fnvolved should lead to'an incréase in the 
guage through the use of a Restatement test rather than perfor- - teaching of critical listening at-a time when educators are pri- 
ee sei pos ng peteehe ee Sy Re reae 3 marily concerned with accountability and with measurable be- 
S) oun 8 language functioning. 
TOE. ove Order No, 77-5168, 65 pages. ° havioral objectives. Order No, 17-16,443, 211 pages. 
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TRAINING PRE-SERVICE TEACHERS IN USE OF STAN- 
DARDIZED LANGUAGE| ARTS ACHIEVEMENT TEST: 
AN EXPERIMENTAL PROJECT 


Pre-service elementary language arts teachers given pro- 
cess consultation training and those given only training 


SHANK, Kathlene Stapfer, Ph.D, 
University of Ulinois at Urbaria-Champaign, 1976 

This study is a statistical study of the training of pre- 
‘ gervice' elementary language arts teachers in use of standard- 
ized language arts achievement tests. The pre-service ele- 
mentary teachers composing the study population were given 
- meaningful and relevant measurement experiences. Each ad- 


a ministered a standardized language arts achievement test, 


“ny SaEeE I ast and then were given the opportunity to plan, - 
rganize, execute and evaluate a language arts program in an 
elementary Classroom for a minimum of nine calendar weeks, 
Two modes of intervention were employed: training and pro- 
cess consultation combined with training. Training was given 
to two-thirds of the population. This training was directed at 
, increasing measurement competencies. Demonstration of the 
* competenctes was encouraged and observed with additional  . 
guidance provided to one-half of the two-thirds receiving 
training. 

The statistical data collected was analyzed to ascertain the 
effectiveness of the modes of intervention on the pre-service 
teachers’ performance of evaluative role behaviors. : The af- 
fects of the modes of intervention on pre-service teachers’ at- 
titudes was also statistically scrutinized. Process consultation 
and training were found to have affected and effected: objec- 
tively measurable knowledge of standardized achievement test, 
attitudes, and activities. « ~ 

Increased objectively measurable knowledge of standardized 
« achievement tests was correlated with the training and process 
consultation given to pre-service elementary language arts 
teachers. The group given neither training or process consul- 
tation did not demonstrate increased objectively measurable 
knowledge of standardized achievement test. 

Training only and process consultation and training did af- 
fect attitudes. ‘Affected attitudes were those toward the value 
of teats, kinds of information, and educational viewpoints to- 
ward worthwhile language arts content and value of evaluation, 

After being given onlv training. pre-sefvice elementary lan- 

guage arts teachers held a quizzical attitude toward the value of 
tests. Training and process consultation combined resulted in 
pre-service teachers adhering more nearly to an over-rejecting 
attitude toward the value of tests. The group givén neither 
training or process consultation tended toward extremism in 

the form of an over-rejecting attitude toward the value of tests. 

Process consultation and training and training only did re- 
bult in pre-service elementary teachers believing more infor- 
mation was so important that they would obtain it. The pre-ser- 
vice elementary teachers given training and process consultation 
desired the greatest numbers of kinds of information, The group 
given neither training or process consultation felt significantly 
fewer kinds of information were important enough to obtain, 

Process consultation and training was affective in causing 
pre-service elementary language arts teachers to strongly agree 
that evaluation can help a teacher see how effective she is in 
the area of language arts. Process consultation and training 
also resulted in pre-service elementary language arts teachers 
strongly agreeing that to‘be worthwhile, language arts content 
must be practically oriented, 

Training and process consultation were successful in stim- 
ulating initiatory activities, discussion of test resuits with stu- 
dents, gathering of information about students, and cosmetically 
oriented activities. Activity stimulated activity, with the most 
activity, process consultation and training, generally stimulat- 
ing the hmost activity. 


~ 


<8 


when compared to previous prerservice elementary teachers 


spent more time gather 
, more, discussed test resu 
- and other educators, utilized 
employed,a greater diversity of.approaches, and used 
ized test results for more pu 

‘teachers given neither training or 
compared with previous 
gual time and did each 


information, utilized test results 
8 more with cooperating teachers 
greater “ariety of materials, 


s. Pre-service elementary 


pre-service 
of the above only equally as well, 


BFFECTS OF THE CULTURAL CONTEXT OF LANGUAGE 
ON THE COGNITIVE PERFORMANCE OF BLACK STUDENTS 


SIMPKINS, Charlesetta, Ed.D, 


, University of Massachusetts, 1977 


\ 


Director: Professor Ellis Olim 


The purpose of this study was to ascertain the effects of the 
cultural context of language on the cognitive performance of 
Black inner-city students. More specifically, this study was 
conducted to test the general hypothesis that when Black non- 
“mainstream students (also referred to as inner-city students) 
are tested for their comprehension of reading materials in 
Standard American English, they will score significantly higher 
when the instrugtions for the lesson are presented in Black 
Vernacular than when the instructions are presented in Standard 
American English, 

From this general hypotbesis the following specific, empir- 
‘cal hypotheses ;e e ge.ératec: 1, When Black non-mainstream 
students are instructed in Black Vernacular on how to dedice 
the meaning of unfamiliar words and phrases from clues in 
reading material, they will score significantly higher, when 
tested in Standard American Englistr on their comprehension 
of the material, than when the instructions are given in Stan- 
dard American English, 2. When Black non-mainstream stu- 
dents are instructed in Black Vernacular on how to deduce the 
meaning of figurative language from clues in reading material, 
they will score significantly higher, when tested in Standard 
American English, on their comprehension of the material, 
than when the AnSvenogions were given in Standard American 
English. 

The Simpkins Test of Cultural Context (STCC) was used as 
the assessment instrument. The STCC wis developed to assess 
the effect ofthe cultural context of language on the performance 
of two cognitive tasks. The two cognitive tasks, Meaning 


Context and Figurative Language e aré incorporated into the 
STCC as subtests. 


The review of the literature was concerned with examining 
two conceptual models, the difference and the deficit models, 
which offer explanations for the poor test performance of Black 
students in the public schools. In addition, the assumptions of 
intervention programs which have emerged from the wey 
were examined. 

A total of 234 Black students inthe seventhand eighth grades 
participated in the study. The students were recruited from, 


~ 


‘ 


bi 
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three schools in the Detroit Public School System which ser- 
ced Black inner-city communities, 

‘i Two one-way analyses of variance were performed on the 

data for the total sample and for each of.the three schools par- 

ticipating in the atudy. In each case, one analysis was per- 

formed on the Meaning from Context subtest and one on the 


Figurative Language subtest with the instructional language 


Black Vernacular vs. Standard American English) serving as_— 
dent variable. 
uel paneer results for the total’sample were significant. 
On the Meaning from Context subtest, the total sample obtained 
a mean score of 10.195 when the explanations were given in 
Standard English as compared to a mean score of 11.613 on the 
subtest with explanations in Black Vernacular (p < .091, one- 
tailed). On‘the Figurative Language subtest, the total sample 
ed a mean score of 5.585 when the explanations were 
given in Standard English as compared to a mean score of 6.544 
on the subtest with explanations in Black Vernacular (p .012, 
tailed). , ; 
om wo —— conclusions may be drawn from this study: 
1, Black non-mainstream students appear to be penalized by 
the failure of most instructional materials to incorporate the 
most familiar language in the explanations and instructions of 
cognitive tasks which they are expected to perform. -2, When 
the language used in tests and other instructional materials 
are modified to be more compatible with the cultural-linguistic 
environment with which Black non-mainstream children are 
familiar, they will score significantly higher on these coer : 
Suggestions are made for further research, utilizing oe 
tional cognitive. tasks, to ascertain to what extent the results 
of this study can be generalized to the learning of cognitive 
tasks in generat. * Order No. 77-21,510, 111 pages. 


THE RELIABILITY OF RATINGS ON THE ESSAY PORTION 
OF THE LANGUAGE SKILLS EXAMINATION : 


SINGLETON, Doris Jacobs, EdD. 
University of Georgia, 1976 


Major Professors: Robert Rentz and Louis Bashaw 


The objective of this study was to investigate the reliability 
-of ratings assigned on the essay portion of the Language Skills 
Examination. This examination is administered to test the 
réading and writing skills of students attending institutions in 
the. University System of Georgia. ‘. 

Four statistical procedures were applied to four different 
samples of essays to investigate the reliability of the essay 
ratings. Reliability estimates were presented in the context 
of the following statistical analyses: percentage estimates of 
rater agreement, product-moment correlations between two 
sets of fatings, intraclass correlational procedures suggested 
by Ebel, and an intraclass correlational procedure involving 
calculations taken directly from a contingency table of all pos- 
sible score combinations’ and their relative frequencies, Sub- 
sequent interpretations of the reliability coefficients were based ‘ 
upon the specific context, point of view, and data used for com- 
puting each reliability coefficient. ; 

The results of this study indicate that the essays written‘on — 
the Language Skills Examination are being reliably scored for 
the intended purposes of thé test. Error was reduced in the 
rating process by having*trained raters who concentrated on 
global ratings and by including multiple ratings in the final 
score, Order No, 77-12,416, 152 pages. 


EVALUATION OF FLORIDA SECONDARY. ENGLISH 
TEACHERS 


SUTTON, Gary-Alan, Ed.D. al 
The University of Florida, 1977 


Chairman: Vincent McGuire 
Fn | 
This study was undertaken to examine the current state of 

faculty evaluation procedures as they applied to English teach- 
ers in Florida’s public secondary schools. yearly evalua- 
tions are a part of Florida law, the study focused'on whether 
these were being carried out according to the stated purpose 
and the letter of the law. The evaluation instruments from 
fifty-five of the sixty-seven state school districts were ana- ° 
lyzed and compared. Results showed a great variety of eval- 
uation systems, from one-page forms for, evaluators to list 
strengths and weaknesses, to sophisticated batteries of forms 
to be used by administrators, teachers, Students, and parents 
in a joint effort to assess faculty performance. Only a single 

. district had a form deSigned specifically for English teachers, 
All agi used one instrument to evaluate teachers in all 

Surveys were gathered from 100 secondary school English 

teachers from a broad representative sample of public schools. 

~The, instructors reported on the current state of faculty evalua- 
tion and made suggestions for improvement. Only 4% reported 
clear violations of the law due to an absence of any evaluation 
during the 1975-1976 school year. However, 15% reported 
violations due to their lack of awareness of the criteria which 
were to be used in their evaluations. The law requires such 
criteria be mad¢ known to the teacher ih advance. 

While the letter of the law was followed in most cases, no 
sincere effort at a true self-improving type of evaluation was 
found in the majority of schools contacted. Teachers rted 
evaluators with expertise in the subject area in only 32% of the 
cases. One or two very brief observations was the rule|in the 
preponderance of cases-studied. Only 36% of the sample were 
satisfied with current evaluation procedures. A total of! 72% 
rated the theory of faculty evaluation from beneficial to ex- 
tremely beneficial. When asked to rate the-actual practice of 
faculty evaluation as they experienced it, the percentage rating 
evaluations as beneficial to extremely beneficial dropped to 
forty-one. S fe 

Teachers stated vhat more observauong were needed foi 
proper evaluations. They feit longer observations were needed, 
asked to be observed by someone with expertise in the area of 
English, and felt that the process should be taken more seti- 
ously. Teachers expressed concern that tenured faculty mem- 
bers were not as accountable as those who might lose jobs due 
to poor ratings. Several selected factors such as school size, 
district population, and the like were analyzed to see if any 
relationship could be seen among these factors and results of 
faculty evaluations. Lower paying districts were consistently 
rated-lower by teachers. Major strengths and weaknesses of 
the instructors, as revealed by the evaluations, were noted. 

Secondary English teachers in Florida indicated their belief 
in faculty evaluation. Their suggestions for the improvement 
of the process were included in this study. : P 

Order No, 77-25,957, 149 pages. | 
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“DETERMINING THE INTERNAL CONSISTENCY OF ENGLISH 
COMPOSITIONS USING SELECTED CRITERIA 


WYNN, James Howard, Ph.D. 
The Louisiana State University and ~ 
Agricaltural and Mechanical College, 1977 


Supervisor: Professor Leonard L. Kilgore 


The major purpose of this study was to determine how large 
- a sample should be taken in order to reliably estimate student ° 
syntactical ability by using T-unit related criteria. Composi-- 
tions Were collected from twenty-nine students in the seventh 
grade and from thirty students in the tenth and twelfth grades | ¢ 
and from thirty college upperclassmen. The mean number of 
words per T-unit, subordinate clauses per T-unit, clauses per 
: Teanit, and T-units per sentence was computed for each grade 
and for all grades combined. The mean number of words per 
T-unit for the seventh grade was correlated with the mean | 
. number of words of the first T-units of ‘all the seventh grade 
compositions. The mean number of words per T-unit for the . 
. + geventh grade was next correlated with the mean of two means: 
the mean number of words for the first and second T-units ° 
from all seventh grade compositions. The mean of three means 
we used for a third correlation. The correlation procedure 
was repeated to encompass ever-increasing sample sizes from 
the grade’s compositions. Correlation coefficients were re- 
_ “corded and plotted on a graph. The correlation prucedure was 
"applied in the same manner for the four T-unit criteria to 
~ grades seven, ten, twelve, and to the college students as well 
as a combination of the four grades. oP , 
The results of the study showed that the small samples 
correlated, in the .80’s and .90's with the mean of the whole at 
twenty T-units or twenty sentences, indicating that a reliable 


estimation of a group’s skill regarding T-unit length, use of 

clauses, and T-units per sentence can be made by using a 

mean of the first twenty T-units or the first twenty sentences 
he compositions in the group. 

er chil r ~ Order No, 77-25,407, 65 pages. 
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